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1.  Objectives 

The  overall  objective  of  our  work  has  been  to  characterize  and  develop  models  for  the 
processing  associated  with  Small  Target  Movement-Detecting  (STMD)  neurons  in  insects, 
building  on  the  results  of  our  prior  AFOSR  research  contract  (F49620-01-C-0030). 

Characterization,  as  the  objective  of  the  experimental  neuroscience  element  of  the 
work,  proceeded  with  the  twin  goals  of  establishing  a  catalogue  of  basic  physiological 
characteristics  and  a  tentative  classification  of  the  various  STMD  types  encountered;  and 
throwing  light  on  particular  aspects  of  the  computations  underlying  their  response 
capabilities.  Modeling  and  simulation  were  undertaken  with  the  goal  of  understanding 
those  computational  mechanisms,  including  the  contributions  of  various  processing  stages 
hypothesized  to  lie  on  the  neural  pathway  leading  to  STMDs. 

2.  Summary  and  Final  Status  of  Effort 

2.1.  Modeling  and  Simulation 

Although  the  study  of  STMDs  remains  in  its  infancy  and  the  goal  of  understanding 
their  computational  basis  is  not  yet  fully  realized,  our  modeling  efforts  have  helped  to 
narrow  our  hypotheses  and  to  direct  the  experimental  questions  that  must  be  answered  in 
further  pursuit  of  the  goal.  In  addition  to  STMD  modeling,  solid  progress  has  been  made  in 
some  new  areas  that  were  not  anticipated  at  the  start  of  the  project  -  in  particular,  the 
modeling  of  rectifying  transient  cells  as  a  possible  STMD  presursor,  and  the  electrical 
properties  of  NMD  A  receptors  as  a  neural  mechanism  for  nonlinear  facilitation.  These  new 
topics  we  believe  will  ultimately  contribute  to  the  understanding  of  STMDs. 

At  the  commencement  of  the  project,  we  posited  a  hierarchy  of  computational  stages 
that  was  consistent  with  available  data  on  STMDs.  As  experimental  work  proceeded, 
plausible  and  detailed  models  for  some  of  these  stages  were  elaborated  only  to  be 
discounted  by  results  from  a  subsequent  experiment.  In  addition,  other  promising  ideas 
were  developed  that  could  not  be  successfully  generalized  or  tied  to  the  electrophysiology. 
As  the  project  progressed,  the  pursuit  of  alternative  hypotheses  led  to  the  consideration  of 
rectifying  transient  cells  and  NMDA-mediated  electrical  behavior  in  neurons.  This  new 
work  has  suggested  that  our  hypothetical  computational  stages  may  not  in  fact  correspond 
to  a  set  of  distinct  processing  elements,  i.e.,  a  succession  of  neuron  types  in  the  pathway 
leading  to  STMDs,  although  we  still  regard  these  stages  as  useful  for  conceptualizing  the 
basic  functional  characteristics  of  STMD  processing. 

Beyond  the  primary  goal,  modeling  and  simulation  have  also  proven  useful  and 
justified  in  their  own  right  as  a  tool  in  support  of  electrophysiological  research: 
experimental  design  has  been  guided  to  a  significant  degree  by  the  testable  hypotheses 
generated  by  modeling  at  both  the  conceptual  and  numerical  levels. 

Finally,  silicon  modeling  -  the  implementation  of  STMD  or  related  models  in  analog 
VLSI  circuitry  —  was  specified  as  an  activity  in  the  project  proposal,  but  was  de- 
emphasized  during  its  execution  in  favor  of  more  detailed  development  of  biologically- 
based  models.  A  single  submission  of  two  test  die  containing  bio-inspired  circuits  was 
made  to  a  DARPA-sponsored  fabrication  run  of  an  experimental  three-dimensional  (three 
interconnected  wafer  tiers)  integrated  circuit  process.  Unfortunately,  delays  in  the 
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completion  of  this  fabrication  have  precluded  test  and  evaluation  of  these  circuits  during 
the  project,  in  spite  of  an  extension  of  the  period  of  performance  of  the  grant. 

2.1.1.  Background 

As  noted,  at  the  close  of  our  prior  AFOSR  contract  on  STMD  neurons  we  hypothesized 
a  hierarchy  of  functional  elements  for  STMD  processing.  This  hierarchy  may  be 
summarized  as  follows: 

1.  Elementary  motion  detection:  A  fundamental  and  local  computation  that  is 
distributed  retinotopically  across  the  entire  visual  field,  and  provides  the  ability  to 
discriminate  local  motion  cues  from  changes  in  luminance  that  are  non-motion- 
related. 

2.  Longitudinal  target  size  discrimination:  The  ability  to  discriminate  ‘smallness’  of 
target  in  direction  of  motion.  Such  processing  may  serve  to  filter  the  outputs  of 
elementary  motion  detectors  for  events  consistent  with  passage  of  a  small  object. 

3.  Target  /  background  speed  discrimination:  The  capability  to  distinguish  a  moving 
target  from  a  moving  cluttered  background  (which  may  contain  small  features 
indistinguishable  from  ‘targets’),  on  the  basis  of  relative  velocity. 

4.  Lateral  target  size  discrimination:  The  ability  to  discriminate  ‘smallness’  of  target 
in  the  dimension  perpendicular  to  direction  of  motion.  This  (along  with 
enhancement  of  directional  selectivity)  was  assumed  to  be  a  principal  function  of 
cells  that  we  previously  labeled  Elementary  Small  Target  Movement  Detectors 
(ESTMDs). 

5.  Wide-field  enhancement  of  small  moving  target  selectivity:  A  postulated 
characteristic  of  wide-field  STMD  neurons  that  exploits  the  constraint  of  continuity 
of  motion  over  some  distance  in  the  receptive  field,  in  order  to  obtain  more  reliable, 
higher-confidence  detection  of  small  moving  targets. 

For  each  of  these  functions,  we  developed  conceptual  (and  in  most  cases, 
computational)  models  during  this  prior  effort.  Following,  we  summarize  these  models: 

1.  Elementary  motion  detection:  The  correlational  or  Reichardt  EMD  model,  or  its 
constituent  parts  (i.e.,  correlator  units),  were  regarded  as  the  primary  candidates  for 
front-end  processing. 

2.  Longitudinal  target  size  discrimination:  A  model  for  this  function  was  based  on  the 
detection  of  triphasic  waves  in  the  outputs  of  individual  Reichardt  EMDs.  With 
proper  pre-processing  in  the  early  visual  pathway  (i.e.,  ‘imperfect’  highpass 
temporal  filtering),  the  presence  of  such  waves  was  found  to  be  an  indicator  of  a 
longitudinally  small  target.  A  computational  model  was  developed,  based  on  gating 
the  rectified  response  to  the  negative-going  output  phase  by  the  local  average  of  the 
output  (which  is  net  positive  for  a  small  target  moving  in  the  preferred  direction). 

3.  Target  /  background  speed  discrimination:  For  this  function,  we  postulated  an 
element  called  a  primitive  small  target  movement  detector  (PSTMD).  This  model 
involves  a  comparison  of  the  traversal  across  pairs  of  ommatidia  of  ‘small  target 
events’  with  the  expected  traversals  of  moving  background.  The  comparison  is 
achieved  with  an  EMD-like  element  that  is  tunable  by  feedback  representing  the 
state  of  local  background  motion,  which  performs  ‘anticorrelations’  of  expected 
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and  actual  events  (by  the  mechanism  of  shunting  inhibition)  rather  than 
correlations.  In  this  way,  stimuli  that  are  consistent  with  background  motion  are 
suppressed,  while  inconsistent  events  -  e.g.,  the  passage  of  a  small  target  with  a 
velocity  different  than  background  -  elicit  a  response. 

4.  Lateral  target  size  discrimination:  For  this  function,  we  developed  a  series  of 
models  for  the  experimentally-observed  characteristics  of  ESTMD  neurons.  These 
are  based  on  a  receptive  field  with  a  notched  inhibitory  surround  and  an  excitatory 
center,  and  ‘tonic’  or  slowly  decaying  responses  in  each  region.  Only  when  a  target 
enters  the  receptive  field  via  the  notch,  and  is  small  enough  laterally  to  fit  through 
the  notch,  can  the  cell  reach  a  net  state  of  excitation.  Silicon  as  well  as 
computational  models  for  this  function  were  developed. 

5.  Wide-field  enhancement  of  small  moving  target  selectivity:  For  this  function,  we 
postulated  a  mechanism  based  on  dendritic  processing.  Excitatory  synapses  from 
earlier  processing  elements  such  as  ESTMDs  are  supposed  to  map  onto  dendrites 
of  a  wide-field  cell  in  a  retinotopic  fashion,  such  that  reinforcement  of  excitatory 
potentials  occurs  when  ESTMDs  are  excited  sequentially  along  continuous  tracks 
in  visual  space.  A  silicon  model  was  also  developed  for  this  processing. 

The  processing  represented  by  these  models  was  assumed  to  proceed  hierarchically  in 
the  order  above,  in  a  series  of  stages  associated  with  the  neural  pathway  that  extends 
between  the  retina  and  the  lobula.  This  overall  hierarchical  model  was  guided  by  the 
assumption  that  successive  steps  should  increasingly  filter  out  events  in  the  input  signal 
that  are  inconsistent  with  small  target  motion  relative  to  background.  By  largely 
eliminating  responses  due  to  flicker,  to  longitudinally  extensive  objects,  and  to  background 
motion,  the  first  three  stages  above  reduce  the  probability  that  a  non-small-target  event  can 
trigger  a  response,  or  inhibit  response  to  an  actual  small  target,  in  an  ESTMD. 

2. 1.2.  Simulation  Methods  and  Data 

Simulations  of  models  for  various  stages  of  STMD  processing  were  performed  during 
the  project.  These  included  individual  stages  in  isolation,  and  in  other  cases  multiple  stages 
cascaded  in  the  order  in  which  it  is  assumed  they  occur  in  the  neural  pathway.  In  all  cases, 
input  data  corresponding  to  raw  luminance  histories  were  passed  through  a  model  for  early 
visual  processing  prior  to  the  models  under  investigation.  Simulations  generally  comprised 
calculation  of  the  time-domain  response  of  a  model  system  to  sequences  of  stimuli.  The 
model  systems  and  stimuli  covered  a  range  of  topologies  and  complexities,  including  some 
characterizing  transient  responses  of  a  single  stage  or  cascade  of  stages  excited  by  a  single 
input  signal,  some  involving  one-dimensional  sensor/processor  arrays  with  lateral 
interactions,  with  a  small  target  moving  against  a  stationary  or  moving  linear  background, 
and  others  using  full  two-dimensional  arrays  and  input  data,  generated  as  described  in 
Experimental  Methods  in  Section  3.  The  data  sequences  for  this  last  class  included  orbiting 
targets  of  two  contrasts  and  sizes,  against  slower  horizontal  and  vertical  background 
motion,  and  are  identified  in  this  report  by  the  labels  ‘otlhcbrs’  (orbiting  target,  large  /  high 
contrast,  background  rightward  slower  than  target),  ‘otlhcbds’  (orbiting  target,  large  /  high 
contrast,  background  downward  slower  than  target),  and  ‘otslcbds’  (orbiting  target,  small  / 
low  contrast,  background  downward  slower  than  target).  The  bulk  of  the  simulations  were 
performed  with  ‘otslcbds’  since  it  is  the  most  challenging  of  the  three  scenarios.  For  these 
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simulations,  data  sampling  and  retinotopic  arrays  of  processing  elements  were  defined  on  a 
hexagonal  lattice  corresponding  to  the  ommatidial  geometry  in  the  insect  eye,  and  a 
retinotopic  architecture  was  maintained  through  the  level  of  the  ESTMD. 

In  addition,  during  the  latter  part  of  the  project,  two-dimensional  input  data  were 
generated  corresponding  to  small  targets  fixed  against  natural  backgrounds,  to  investigate 
responses  when  the  background  was  animated  and  there  was  no  relative  motion  between  it 
and  the  targets  themselves.  In  the  earliest  of  these,  data  and  processor  arrays  were  defined 
on  a  rectangular  lattice,  but  the  models  were  subsequently  transferred  to  a  hexagonal 
geometry.  The  corresponding  simulations  primarily  involved  models  for  the  rectifying 
transient  cells  and  precursor  processing. 

Models  were  coded  and  simulations  were  performed  using  the  tool  SPICE  (Simulation 
Program  with  Integrated  Circuit  Emphasis,  as  the  product  T-SPICE,  Tanner  Research, 
Monrovia,  CA),  and  in  the  latter  part  of  the  project,  MATLAB  /  Simulink  (The  Math 
Works,  Natick,  MA).  Details  of  the  imagery  generation,  data  handling,  and  data 
interchange  formats  followed  the  conventions  described  in  reports  for  our  prior  project 
F49620-0 1  -C-0030. 

2.1.3.  Early  Vision  (O’Carroll,  Brinkworth,  Shoemaker) 

During  the  project,  we  adopted  a  standard  model  for  early  visual  processing  that  is 
consistent  with  the  known  properties  of  the  compound  eye  optics,  retina,  and  first  optic 
ganglion  (the  lamina)  in  insects.  This  model  involves  no  contribution  of  our  own,  but  is 
based  on  prior  work.  A  reasonable  representation  of  early  vision  was  regarded  as  important 
for  the  fidelity  of  simulations  of  subsequent  processing.  Similar  processing  was  used  in 
some  of  the  simulations  during  the  prior  contract,  but  the  current  version  was  applied 
systematically  to  the  input  (i.e.,  luminance)  data  used  in  the  present  effort.  We  detail  the 
model  here  as  follows. 

The  geometrical  optics  of  the  model  were  scaled  to  reflect  the  approximate 
characteristics  of  the  eyes  of  large,  visually  acute  dipterans.  The  hexagonal  grid  was  set  up 
such  that  the  orbiting  target  in  the  input  scenarios  subtends  about  15  inter-ommatidial 
angles  in  visual  space.  This  is  a  slightly  closer  view  than  would  be  observed  by  an  insect 
mounted  in  the  experimental  stage  in  the  laboratory,  viewing  the  original  full-size  video 
projected  on  the  CRT.  Compound  eye  optics  were  modeled  by  blurring  of  relatively  high- 
resolution  input  imagery  by  a  Gaussian  modulation  transfer  function  F  of  the  form 

F(</>)  =  exp[-2.11</>2  /a2],  (1) 

where  (f>  is  angular  deviation  from  the  optical  centerline  of  an  individual  ommatidium,  and 
the  standard  deviation  a  was  set  to  about  1.1  times  the  inter-ommatidial  angle.  Discrete 
spatial  convolution  with  this  function  was  performed  on  the  original  pixel  grid  during  the 
process  of  resampling  of  the  raw  input  data  onto  the  hexagonal  grid,  in  order  to  form  input 
data  for  the  simulations. 

The  early  vision  model  included  a  linear  lowpass  temporal  filter  that  reflects 
photoreceptor  temporal  response,  with  a  transfer  function  (Laplace  transform)  l/(np+l), 
where  s  is  the  Laplace  variable,  and  where  tL  was  set  in  the  neighborhood  of  3ms, 
corresponding  to  a  comer  frequency  of  about  53Hz.  Following  this,  a  Lipetz,  or  Naka- 
Rushton  transformation  was  applied.  This  transformation  is  specified  by  the  equation 


Tanner  Research  and  the  University  of  Adelaide 


5 


Qrant  FA9550-04- 1-0283 


Final  Report  31  DEC  2007 


U  =  Ia  /(la  +I0a  ),  (2) 

where  I  is  the  input  intensity,  I0  a  parameter  defining  mid-response  level,  a  is  an  exponent 
between  0.5  and  1,  and  U  is  the  output.  The  exponent  a  was  set  to  the  value  0.7.  In  the  first 
half  of  the  project,  the  mid  response  level  parameter  I0  was  set  to  120,  which  approximates 
the  geometric  mean  of  the  8-bit  luminance  data  (range  [0,255])  in  the  imagery  used  in 
simulations.  In  the  latter  part  of  the  project,  this  parameter  was  varied  by  setting  it  to  a 
low-pass-filtered  version  of  the  input  luminance,  with  a  time  constant  on  the  order  of  Is,  as 
a  means  of  approximating  luminance  adaptation.  Finally,  a  highpass  temporal  filter  with 
transfer  function  (Laplace  transform)  ths/(ths+1)  was  applied,  where  s  is  the  Laplace 
variable,  and  where  rH  was  set  various  values  depending  on  the  detail  of  the  individual 
models  to  follow. 

2. 1. 4.  Elementary  Motion  Detection  and  the  Correlational  EMD  (O  ’ Carroll 
Shoemaker) 

Following  up  on  work  under  the  prior  contract,  we  performed  further  simulations  in 
which  the  correlational  or  Reichardt  EMD  was  used  in  combination  with  detection  of 
triphasic  output  signatures  for  elementary  motion  detection  and  longitudinal  size 
discrimination.  These  included  simulations  using  a  detailed  model  of  the  EMD  as 
implemented  in  silicon  in  the  prior  contract.  This  approach  was  abandoned  for  subsequent 
models  including  the  Small  Event  Motion  Detector  and  Rectifying  Transient  Cell  models. 

2.1.5.  The  Small  Event  Motion  Detector  (Shoemaker) 

The  Small  Event  Motion  Detector  (SEMD)  model  originated  during  the  present  grant. 
This  element  provides  both  elementary  motion  detection  and  selectivity  for  longitudinally 
small  objects  in  a  single  unit.  It  is  computationally  similar  to  the  standard  correlational 
elementary  motion  detector  (EMD)  in  that  the  fundamental  operation  is  the  product  of 
input  signals  from  neighboring  ommatidia.  In  the  SEMD,  however,  these  input  signals  are 
first  passed  through  a  highpass  filtering  stage  with  a  short  time  constant,  so  that  the  signals 
that  are  multiplied  together  reflect  rapid  transients.  The  multiplication  operation  is 
followed  by  half-wave  rectification  for  the  negative-going  phase  of  the  product.  These 
operations  select  for  transients  that  are  of  opposite  sign  at  the  two  neighboring  locations, 
and  render  the  SEMD  sensitive  to  the  coincidence  of  a  leading  edge  at  one  location  and  a 
trailing  edge  at  another  -  an  effective  discriminant  for  the  passage  of  a  longitudinally  small 
target.  Because  no  delay  function  or  other  asymmetric  temporal  processing  is  associated 
with  the  two  legs  of  the  correlator,  the  response  at  any  particular  speed  is  maximal  when 
the  leading  and  trailing  edges  are  exactly  one  pixel  apart  (i.e.,  the  optimum  target  extent  is 
one  ommatidium).  Another  consequence  of  this  symmetry  is  that  the  SEMD  has  no 
directional  sensitivity  -  the  response  is  the  same  if  the  direction  of  motion  of  a  target  is 
reversed.  It  does  of  course  have  orientation  selectivity  on  a  two-dimensional  ‘retina’  by 
virtue  of  being  aligned  with  some  particular  inter-ommatidial  axis.  There  is  one  SEMD  for 
each  inter-ommatidial  boundary  in  our  model  system. 

During  the  first  half  of  the  project,  considerable  effort  was  directed  toward 
development  and  evaluation  of  this  model.  Initially,  it  was  simulated  in  isolation  using 
synthetic  input  data  corresponding  to  a  moving  pulse  against  a  uniform  background.  Its 
basic  operation  was  investigated,  and  the  consistency  of  its  joint  target  speed/size  tuning 


Tanner  Research  and  the  University  of  Adelaide 


6 


Qrant  FA9550-04- 1-0283 


Final  Report  31  DEC  2007 


with  the  characteristics  of  biological  STMD  neurons  was  demonstrated.  A  suite  of  more 
realistic  simulations  was  then  performed  to  further  characterize  it  and  compare  its  results  to 
neurobiological  data  from  STMD  cells.  We  studied  the  effects  of  varying  the  tuning  and 
topology  (i.e.,  time  constant  and  order)  of  the  highpass  filter,  in  simulations  of  single  units 
and  small  subarrays.  These  simulations  used  segments  of  the  orbiting  target  datasets,  each 
containing  a  target  transit  in  one  direction,  or  background  motion  only,  and  employed  the 
standard  early  vision  model  of  Section  2.1.3  above.  Following  these  simulations,  a 
‘standard’  version  of  the  SEMD  was  selected  and  simulated  for  the  various  complete 
orbiting  target  scenarios  in  a  full-size  (31x31  hexel)  array.  The  model  was  ultimately 
abandoned  following  electrophysiological  experiments  to  test  its  predictions,  and  attention 
focused  on  the  rectifying  transient  cells. 

2.1.6.  The  Primitive  Small  Target  Movement  Detector  (Shoemaker) 

The  Primitive  Small  Target  Movement  Detector  (PSTMD),  originated  during  the  prior 
contract,  is  intended  to  discriminate  target  from  background  motion  on  the  basis  of  relative 
velocity.  Its  operation  involves  a  comparison  of  local  events  with  the  expected  traversal  of 
moving  background.  The  comparison  is  achieved  with  an  EMD-like  element  that  is  excited 
by  local  input,  and  is  also  subject  to  shunting  inhibition  by  a  delayed  signal  passed  from  a 
nearby  unit.  In  the  original  version  of  the  PSTMD,  the  delayed  signal  was  simply  derived 
from  the  input  signal  of  a  single  nearby  PSTMD,  and  the  delay  operator  was  a  lowpass 
filter.  This  delayed  signal  represents  the  estimate  of  background  motion,  and  the  delay  time 
is  tuned  to  reflect  the  speed  of  that  motion.  The  shunting  inhibition  is  modeled  with  a 
division,  with  a  small  constant  term  in  the  denominator  representing  the  inverse  of  the 
maximum  gain  for  the  unit.  The  inhibition  is  intended  to  suppress  responses  to  moving 
background  events,  but  to  leave  the  PSTMD  responsive  to  events  inconsistent  with  the 
background  motion,  i.e.,  the  passage  of  a  small  target  with  a  different  velocity.  We 
performed  simulations  of  the  PSTMD  concept  using  the  arrays  developed  in  the  prior 
project,  but  with  inputs  to  PSTMDs  supplied  by  SEMDs  rather  than  correlational  EMDs  or 
synthetic  input  signals  as  used  in  prior  work.  The  SEMDs  were  excited  by  input  segments 
derived  from  the  orbiting  target  scenarios  (some  including  target  transit  and  some, 
background  motion  alone  to  test  rejection  of  background  motion).  In  our  model  system,  a 
full  complement  of  PSTMDs  would  include  one  pair  for  each  SEMD  (i.e.,  one  for  each 
direction  along  each  inter-ommatidial  axis). 

Simulations  with  the  original  model  and  several  variants  confirmed  or  revealed  a 
number  of  shortcomings  of  the  approach  (discussed  in  ‘Accomplishments  /  New  Findings’ 
in  Section  3.1.3  below),  including  in  particular  the  inability  to  distinguish  target  from 
background  motion  when  the  velocities  are  at  all  similar.  As  a  result  we  undertook  a  major 
revision  of  the  PSTMD  model  during  the  present  grant. 

In  this  revision,  we  developed  the  notion  of  a  ‘propagating  predictor’  for  background 
motion.  This  involves  a  retinotopic  network  with  spatial  as  well  as  temporal  memory,  in 
which  estimates  of  SEMD  responses  to  background  motion  are  passed  to  a  ‘neuron’  from 
its  neighbor(s),  reinforced  by  any  local  SEMD  input,  and  then  delayed  and  passed  on. 
Because  the  local  estimate  of  background  motion  is  dependent  on  its  history  across  a 
region  of  the  network,  and  not  just  at  one  neighboring  unit,  we  expected  this  approach  to 
be  capable  of  a  more  accurate  estimate  of  background  events  with  less  influence  due  to  a 
target  that  is  moving  with  a  velocity  similar  to  the  background  (assuming  that  the  delay  in 
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the  network  is  tuned  to  match  the  local  speed  of  propagation).  This  method  is  related  to  a 
target-tracking  algorithm  developed  by  T.  Bartolac  at  Tanner  Research  and  called  the 
Wave  Process  (Bartolac  and  McCarley,  2003),  which  involves  propagation  of  waves  in  a 
two-dimensional  discrete  network  and  the  reinforcement  of  those  waves  by  input  from  a 
moving  stimulus.  Our  version  is  biologically  plausible  and  could  be  realized  in  a  neural 
network. 

This  PSTMD  model  was  developed  initially  in  one-dimensional  form  and  simulated 
using  input  data  segments  extracted  from  columns  of  the  orbiting  target  dataset  (some 
containing  a  target  transit  and  some  containing  only  background  motion).  Simulations 
included  early  vision  and  SEMD  processing  as  well.  Encouraging  results  were  obtained  in 
the  one-dimensional  case,  and  we  attempted  with  considerably  less  success  to  generalize 
the  concept  to  a  two-dimensional  retinotopic  network. 

During  the  latter  part  of  the  grant,  work  on  the  PSTMD  was  put  on  hold  due  to 
electrophysiological  experiments  suggesting  the  importance  of  small  target  spatial  statistics 
for  discrimination  in  biological  STMDs,  and  the  apparent  difficulty  that  they  have 
discriminating  same-direction  target  and  background  motion. 

2.1.7.  The  Elementary  Small  Target  Motion  Detector  (Shoemaker,  O  ’ Carroll ) 

Models  for  the  Elementary  Small  Target  Motion  Detector  (ESTMD)  were  first 
developed  during  the  prior  grant,  and  in  contrast  with  the  SEMD  and  PSTMD,  are  based 
on  the  observed  characteristics  of  a  class  of  neurons  that  have  been  encountered  and 
characterized  electrophysiologically  in  the  past  -  the  cells  also  labeled  ESTMDs.  As  an 
element  in  our  proposed  processing  hierarchy,  ESTMD  models  provide  lateral  target  size 
discrimination  as  well  as  imparting  or  enhancing  directional  selectivity.  These  are  based  on 
a  receptive  field  with  a  notched  inhibitory  surround  and  an  excitatory  center,  and  ‘tonic’  or 
persistent  responses  in  each  region.  The  fundamental  idea  is  that  only  when  a  target  enters 
the  receptive  field  via  the  notch,  and  is  small  enough  laterally  to  fit  through  the  notch,  is 
the  cell  able  reach  a  net  state  of  excitation.  ESTMDs  in  our  prior  work  were  arranged  in 
hexagonal  arrays,  with  a  one-unit  excitatory  center  and  five  of  the  six  surrounding  units 
defining  the  notched  inhibitory  surround.  Input  data  were  synthetic  (i.e.,  not  derived  from 
any  prior  visual  motion  processing  stages). 

Modeling  and  simulation  of  the  ESTMD  during  the  current  grant  focused  initially  on 
the  ‘tonic’  temporal  element  that  is  responsible  for  the  nonlinear  dynamic  behavior  of  the 
overall  model.  We  developed  a  concept  (demonstrated  in  a  silicon  model  near  the  end  of 
the  prior  effort)  in  which  rapid  activation  is  followed  by  a  ‘sustained  tonic’  response  rather 
than  slow  decay  of  the  output  that  commences  immediately  after  stimulation.  We  expected 
this  characteristic  would  ultimately  allow  the  ESTMD  model  to  discriminate  more 
unambiguously  between  preferred-  and  non-preferred-direction  moving  stimuli. 

After  work  on  this  temporal  model,  a  complete  ESTMD  was  formulated  with  inputs 
derived  from  prior  motion  processing  stages  for  the  first  time.  These  ESTMDs  receive 
inputs  directly  from  SEMDs  (PSTMDs  were  not  included  due  to  lack  of  success  in 
formulating  two-dimensional  architectures).  Receptive  fields  are  larger  than  in  the  prior 
work,  but  as  before  contain  inhibitory  and  excitatory  regions,  with  each  showing  persistent 
behavior,  and  with  the  notch’  in  the  inhibitory  subfield  implemented  by  an  asymmetric 
pattern  of  connections  from  inhibitory  ‘intemeurons’  onto  the  ESTMD  output  unit. 
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2.1.8.  Rectifying  Transient  Cells  (Shoemaker,  Wiederman) 

By  the  halfway  point  of  the  project,  pilot  experiments  on  STMD  neurons  with  new 
experimental  protocols  had  provided  feedback  on  some  of  the  hypotheses  involved  in  the 
models  described  above.  Particularly  for  the  processing  elements  for  which  no  neural 
correlates  have  been  observed  electrophysiologically  (i.e.,  the  SEMD  and  PSTMD),  the 
new  data  made  clear  just  how  speculative  various  aspects  of  these  models  are,  leading  to  a 
reassessment  of  the  overall  approach:  while  continued  development  of  ‘bio-inspired’ 
models  might  lead  to  new  algorithms  for  small  target  detection  and  tracking,  their  value  in 
understanding  details  of  the  processing  in  the  biological  system  is  limited  until  detailed 
physiological  evidence  can  be  applied  to  test  their  key  features.  This  provided  motivation 
for  development  of  further  test  protocols,  and  especially  for  the  identification, 
investigation,  and  modeling  of  other  possible  neural  elements  in  the  STMD  pathway. 

For  this  reason,  modeling  and  simulation  at  the  level  of  the  STMD  was  relatively 
limited  during  the  last  half  of  the  project.  We  instead  pursued  an  interest  in  the  Rectifying 
Transient  Cells  as  a  possible  precursor  to  the  STMD.  Initial  modeling  efforts  were  based 
on  previously  reported  results  (Jansonius  and  van  Hateren,  1991,  1993;  Osorio,  1987)  and 
predated  our  own  electrophysiological  work  on  such  cells.  We  developed  a  basic  model  for 
the  lamina  neurons  of  this  class  (the  ‘on-off’  cells  of  Jansonius  and  van  Hateren,  1991, 
1993),  focusing  primarily  on  the  full-wave  rectifying  nonlinearity  and  temporal 
characteristics  (i.e.,  fast  adaptation).  Separate  ON  and  OFF  processing  channels  with 
independent  adaptation  were  proposed,  in  work  done  primarily  by  the  industrial  partner. 
Subsequently,  as  experimental  work  commenced,  the  model  was  elaborated  with  detailed 
spatial  interactions  added  to  its  temporal  features  in  work  by  the  academic  partner.  The 
model  was  then  applied  to  small  target  data,  with  targets  moving  at  the  same  velocity  as  a 
cluttered  background,  to  demonstrate  the  salience  of  small  target  spatial  characteristics  in 
RTC  response.  These  results  have  given  us  reason  to  believe  that  we  may  finally  have 
identified  a  relevant  stage  of  precursor  processing  for  the  STMD,  and  will  be  able  to  apply 
electrophysiological  data  from  these  cells  in  order  to  develop  a  more  successful  and 
biologically  grounded  STMD  model. 

2. 1. 9.  Dendritic  Spatiotemporal  Processing 

Dendritic  spatiotemporal  processing  has  been  postulated  to  enhance  the  reliability  and 
selectivity  for  small  moving  targets  in  wide-field  STMDs.  Excitatory  synapses  from  earlier 
processing  elements,  such  as  ESTMDs,  are  supposed  to  map  onto  dendrites  of  a  wide-field 
cell  in  a  retinotopic  fashion,  such  that  reinforcement  of  excitatory  potentials  occurs  when 
ESTMDs  are  excited  sequentially  along  continuous  tracks  in  visual  space. 

An  abstract  computational  model  and  a  silicon  implementation  were  developed  for 
this  type  of  processing  during  the  prior  contract,  but  efforts  under  the  current  grant  have 
been  limited  to  consideration  of  biophysical  mechanisms  for  the  nonlinear  facilitation  that 
would  underlie  the  hypothesized  fiinction.  This  led  to  work  on  the  implications  of  a 
particular  nonlinear  synaptic  receptor  conductance,  as  described  in  the  following  section. 
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2.1.10.  NMDA  Receptors  and  Bistable  /  Amplifying  Neural  Behavior 
(Shoemaker) 

NMDA  receptors  are  a  class  of  metabotropic  or  second-messenger  synaptic  receptors 
that  are  typically  activated  in  vivo  by  the  neurotransmitter  glutamate,  with  N-methyl-D- 
aspartate  (NMDA)  acting  as  a  specific  agonist.  The  current/voltage  relationship  of  NMDA 
receptor  ion  channels  is  significantly  nonlinear  under  physiological  conditions,  which  can 
give  rise  to  bistable  and  facilitatory  behavior  in  neurons  and  neural  structures.  Such 
behavior  could  provide  a  biophysical  substrate  for  the  sort  of  nonlinear  facilitation 
envisioned  for  dendritic  spatiotemporal  processing  as  mentioned  above,  and  indeed  for  the 
fundamental  nonlinearity  underlying  directional  motion  detection  itself. 

In  the  last  nine  months  of  the  project,  we  analyzed  the  conditions  under  which  bistable 
regimes  of  operation  prevail  in  neurons  and  dendrites,  and  also  general  amplifying 
properties,  due  to  NMDA  receptors  in  concert  with  other  membrane  conductances.  We 
considered  primarily  the  case  in  which  other  conductances  are  nearly  ohmic  in  nature; 
these  include  a  number  of  important  synaptic  conductances  such  as  AMPA,  GABAa,  and 
GABAb  receptors.  The  methodology  consisted  of  numerical  and  mathematical  analyses  of 
both  single  electrical  compartments  and  cable-like  dendrites  with  proximal  somatic  loads, 
with  NMDA  and  other  conductances  present  in  the  membrane.  One  product  of  this  work  is 
models  for  compartments  and  dendrites  that  can  be  used  to  implement  NMDA-based 
computational  primitives  in  further  simulations. 

2.1.11.  Silicon  Modeling  (Shoemaker) 

The  DARPA-sponsored  experimental  three-dimensional  IC  process  (3 DM2)  involves 
stacking  of  multiple  wafers  (three  layers  at  present),  with  electrical  contacts  between 
wafers,  in  a  silicon-on-insulator  process  developed  at  MIT  7  Lincoln  Laboratory.  This 
makes  it  a  very  promising  medium  for  direct  implementation  of  the  sensor-parallel,  layer- 
serial  architecture  that  is  natural  for  neuromorphic  sensory  processing.  We  developed 
visual  motion  detection  circuitry,  including  photodetectors,  early  vision,  and  models  for 
both  rectifying  transient  cells  and  correlational  EMDs.  The  final  version  of  these  circuits 
were  submitted  to  fabrication  on  4  NOV  2006,  but  were  not  be  received  back  before 
termination  of  the  present  grant. 
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3.  Accomplishments  /  New  Findings 

3.1.  Modeling  and  Simulation 

3.1.1.  Elementary  Motion  Detection  and  Longitudinal  Size  Discrimination 
(O’Carroll,  Shoemaker) 

When  a  circuit-based  model  for  the  correlational  EMD  was  simulated  in  response  to 
synthetic  small  targets,  we  found  that  the  triphasic  output  signatures  could  be  obtained,  as 
in  prior  simulations  with  biologically-based  models.  This  result  required  proper 
configuration  and  tuning  of  the  early  vision  filtering  (i.e.,  an  imperfect  highpass 
characteristic  that  passes  some  DC  component),  and  we  found  that  it  was  not  particularly 
robust  with  respect  to  variations  in  the  model  parameters  and  the  characteristics  of  the 
synthetic  target.  Thus,  detection  of  longitudinally  small  moving  features  based  on  this 
signature  was  judged  to  be  a  problematic  approach  (particularly  for  a  VLSI  model),  and 
this  result  motivated  the  investigation  of  alternative  approaches  to  modeling  the  function  of 
longitudinal  target  size  discrimination. 

3.1.2.  The  Small  Event  Motion  Detector  (Shoemaker) 

The  SEMD  represents  an  attempt  to  develop  a  more  robust  model  for  both  elementary 
motion  detection  and  longitudinal  size  discrimination  that  (at  its  conception)  was  also 
consistent  with  the  electrophysiological  data  available.  As  noted,  the  model  is 
computationally  similar  to  the  standard  correlational  EMD:  the  fundamental  operation  is 
the  correlation  (product)  of  input  signals  from  neighboring  ommatidia.  The  SEMD, 
however,  is  preceded  by  a  highpass  filtering  stage  with  a  short  time  constant,  so  that  the 
prefiltered  inputs  produced  by  this  stage  reflect  rapid  transients.  In  addition,  the  correlation 
operation  is  followed  by  half-wave  rectification  for  the  negative-going  phase  of  the 
product.  These  operations  select  for  transients  which  at  any  instant  are  of  opposite  sign  at 
the  two  neighboring  locations,  and  render  the  SEMD  sensitive  to  the  coincidence  of  a 
leading  edge  at  one  location  and  a  trailing  edge  at  another  —  an  effective  discriminant  for 
the  passage  of  a  longitudinally  small  target.  Thus  selectivity  for  longitudinal  smallness  is 
made  an  inherent  part  of  an  elementary  motion  detection  operation. 

Basic  Response  Characteristics  of  the  SEMD 

The  response  of  an  SEMD  to  synthetic  targets  is  illustrated  in  Figure  1  along  with  its 
prefiltered  inputs  to  give  some  intuition  with  respect  to  its  operation.  The  top  graph 
illustrates  the  inputs  and  output  of  an  SEMD  in  response  to  a  moving  target  (consisting  of 
a  square  pulse  of  greater  luminance  than  a  uniform  background)  whose  longitudinal  extent 
is  equivalent  to  the  inter-ommatidial  separation.  The  bottom  graph  illustrates  the  response 
of  the  same  SEMD  to  a  target  whose  longitudinal  extent  is  four  times  the  interommatidial 
distance. 
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Figure  1:  Response  of  an  SEMD  to  the  passage  of  a  small  moving  target  1.5°  in 
longitudinal  extent  (top)  and  6°  in  longitudinal  extent  (bottom).  The  interommatidial 
angle  is  1.5°  and  the  target  speed  32°/s  in  both  cases.  The  first  or  ‘upstream’  input  to 
the  SEMD  is  shown  in  dark  blue,  the  second  or  ‘downstream’  input  in  lighter  blue, 
and  the  output  in  red.  The  time  constant  of  the  highpass  prefilter  is  set  to  40ms  in 
both  cases. 


The  spatiotemporal  tuning  of  the  SEMD  was  characterized  in  simulations  by  recording 
its  peak  output  as  a  function  of  target  speed  and  longitudinal  extent.  This  tuning  is 


Tanner  Research  and  the  University  of  Adelaide 


12 


Qrant  FA9550-04- 1-0283 


Final  Report  31  DEC  2007 


remarkably  similar  to  that  observed  in  biological  STMD  neurons,  with  peak  response 
occurring  at  higher  speeds  for  longer  targets. 


— xt  =  0.75  degrees 
xt  =  1.5  degrees 
— *-xt  =  3.0  degrees 
-♦-xt  =  6.0  degrees 


1  10  100  1000 
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Figure  2:  Spatiotemporal  tuning  curves  for  the  SEMD  model.  The  peak  output  of  the 
SEMD  is  displayed  as  a  function  of  small  target  speed,  with  the  longitudinal  extent  of 
the  target,  labeled  ‘xt’,  as  a  parameter.  Other  parameters  are  the  same  as  in  Figure  1. 


In  the  SEMD  model  there  is  no  delay  filtering  or  other  asymmetry  applied  to  either 
term  of  the  product  operation,  so  the  response  is  not  directional  (i.e.,  while  it  implicitly  has 
an  orientation  since  it  is  aligned  with  the  axis  between  two  particular  ommatidia,  it  is 
symmetric  with  respect  to  motion  in  either  direction  along  this  axis). 

Further  Investigations  of  the  SEMD 

Following  these  basic  characterizations  of  the  model,  we  simulated  the  SEMD  in 
response  to  the  input  data  in  the  orbiting  target  scenarios  to  obtained  further  insight  into  its 
response  to  moving  background  with  ‘naturalistic’  statistics  as  well  as  to  small  targets 
moving  relative  to  such  background.  In  some  of  these  simulations,  as  well  as  by  analysis, 
we  investigated  the  influence  of  the  temporal  characteristics  of  the  SEMD  temporal 
highpass  filter  on  the  function  of  the  model.  The  time  constant  of  the  filter  was  varied 
during  these  simulations,  and  in  addition,  second-order  as  well  as  the  original  first-order 
filters  were  evaluated.  The  conclusions  from  this  phase  of  the  work  is  summarized  as 
follows. 

The  fundamental  mechanism  by  which  the  SEMD  operates  is  the  detection  of  edges 
(rapid  transitions)  of  opposite  slopes  at  neighboring  ommatidia.  Thus  the  filter  must  be  an 
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edge  detector  capable  of  producing  opposite-sign  responses  to  leading  and  trailing  edges  of 
a  small  target,  over  the  range  of  target  speeds  (typically  tens  to  one  or  two  hundred  degrees 
per  second  in  the  biological  system)  and  sizes  (less  than  one  to  two  or  three  inter- 
ommatidial  angles)  of  interest.  This  means  that  the  filter  should  reject  temporal  frequencies 
including  and  lower  than  the  fundamental  associated  with  small  target  passage.  The  issue 
can  be  understood  intuitively  by  considering  the  response  of  a  first-order  highpass  filter  to 
a  pulse  representing  transition  of  a  small  target  across  an  ommatidium: 


Figure  3:  Response  of  first-order  highpass  filter  to  a  simulated  small  moving  target 
input.  An  input  pulse  is  rounded  by  blurring  due  to  compound  eye  optics  and  early 
visual  processing.  The  time  constant  of  the  filter  is  a  parameter. 


When  the  time  constant  is  long,  the  filter  passes  substantial  energy  of  the  pulse 
fundamental  and  is  a  poor  basis  for  an  edge  detector.  As  the  time  constant  decreases,  the 
filter  begins  to  approximate  a  differentiator  (transfer  function  tes/(1+tes)  ~  zEs,  where  zE 
is  the  highpass  time  constant)  and  becomes  a  better  basis  for  edge  detection.  As  the  time 
constant  decreases  further,  however,  the  output  amplitude  falls  significantly.  Clearly,  the 
time  constant  must  be  chosen  to  give  reasonable  edge  detection  over  the  range  of  target 
sizes  and  speeds  of  interest,  that  is,  of  size  1/tt  to  1  / (2 n)  times  the  typical  event  duration  in 
the  center  of  this  range.  When  the  background  motion  behind  the  target  induces  relatively 
large,  fast  changes  in  luminance,  the  time  constant  needs  to  be  chosen  at  the  lower  end  of 
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this  range  so  that  the  background  changes  do  not  prevent  the  filter  from  generating  two 
opposite-sign  transients  in  response  to  target  passage.  We  found  that  for  the  orbiting  target 
data,  in  which  the  events  in  early  vision  outputs  associated  with  target  passage  were 
typically  of  40ms  -  60ms  duration,  a  time  constant  te  =  15ms  gave  good  performance  in 
the  SEMD.  This  is  shorter  than  the  value  arbitrarily  chosen  during  initial  characterization 
(40ms),  and  results  in  a  larger  number  of  zero  crossings  for  the  SEMD  filter  outputs  in 
response  to  our  input  datasets. 

In  examining  the  response  of  the  SEMD  with  a  second-order  highpass  filter,  we  found 
that  we  could  obtain  larger  responses  for  target  relative  to  background  events,  than  could 
be  obtained  with  the  first-order  version.  However,  the  responses  of  the  filter  to  target  edges 
were  generally  multimodal,  due  to  the  fact  that  the  second-order  filter  approximates  a 
double  differentiator  ( tes2/(1+tes )2  ~  rEs2)  for  relevant  values  of  the  time  constant  zE. 
Upon  examination,  the  improvement  in  target-to-background  response  was  found  due  to 
the  sharpness  of  signal  transitions  with  target  passage  as  compared  to  background  motion 
in  our  naturalistic  input  data,  and  in  scenarios  (such  as  urban  environments)  in  which  sharp 
edges  are  numerous,  this  advantage  would  disappear.  In  addition,  the  multimodal 
responses  of  the  filter  complicate  the  detection  of  small  targets  by  the  mechanism  of 
correlation  of  neighboring  responses  and  rectification  for  the  negative-going  phase  of  the 
product.  Consequently,  we  selected  the  first  order  filter  as  the  standard  temporal 
component  for  our  SEMD  model. 

Responses  obtained  for  this  SEMD  model  are  typically  stronger  for  the  target  in  our 
data  sequences  than  for  background  artifacts,  even  for  the  small  /  low-contrast  target,  in 
spite  of  the  fact  that  there  is  no  inherent  speed-based  discrimination  capability  in  the 
SEMD.  Its  responses  must  therefore  be  due  to  the  differences  between  background 
statistics  and  the  target  characteristics,  since  both  are  moving  at  ‘reasonable’  and  not  too 
different  speeds.  Below  in  Figure  4  is  the  output  of  a  vertically-oriented  SEMD  at  the  ‘3- 
o’clock’  position  on  the  orbit,  where  the  background  and  target  directions  are  the  same 
(downward).  This  is  the  small  /  low-contrast  target,  which  transits  the  SEMD  just  before  1 , 
3,  and  5  seconds: 
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t  igure  4.  Response  of  an  SEMD  to  moving  input  data,  with  a  small  target  transiting 
the  unit  just  before  times  Is,  3s,  and  5s,  in  the  same  direction  (downward)  as 
background,  and  with  the  SEMD  axis  also  vertically  oriented.  The  target  is  small  / 
low-contrast  (scenario  ‘otslcbds’). 

Interestingly,  the  difference  between  target  and  background  responses  is  not  as 
pronounced  when  the  target  moves  in  the  opposite  direction  from  the  background  (i.e.,  at 
the  ‘9-o’clock’  position  on  the  orbit),  at  least  with  this  particular  data  sequence.  This  may 
be  due  to  ‘collision’  with  opposite-direction  events.  However,  relative  velocity  would 
provide  a  powerful  additional  discriminant  in  the  case  of  opposing  motion,  and  target 
could  easily  be  picked  out  from  background  on  that  basis  by  subsequent  processing. 

In  the  movie  segments  semd_array_otlhcbrs.avi  and  semd_array_otslcbds.avi 
submitted  on  CD  media  as  supplements  to  prior  progress  reports,  the  response  of  the 
standard  SEMD  model  to  two  of  the  orbiting  target  scenarios  is  shown  for  full-sized 
SEMD  arrays.  These  movies  use  the  same  color  map  to  indicate  intensity  as  has  been  used 
on  prior  movies  for  this  project.  In  both  video  segments,  diamond-shaped  metapixels  are 
used  to  display  the  SEMD  outputs,  reflecting  the  fact  that  there  is  an  SEMD  associated 
with  each  inter-ommatidial  boundary.  Each  diamond  is  a  tessellation  in  which  the  screen 
pixels  are  grouped  together  about  the  closest  midpoint  on  the  boundary  between  two 
hexagons,  and  represents  an  SEMD  that  operates  on  the  outputs  of  early  visual  processing 
for  the  corresponding  ommatidia.  The  orientation  of  the  long  axis  of  each  diamond  reflects 
the  orientation  of  the  corresponding  SEMD.  The  movie  sejnd_array  otlhcbrs.avi  shows 
the  response  to  the  large/high-contrast  target  scenario-  ‘otlhcbrs’,  and 
semd_array_otslcbds.avi  the  response  to  the  small/low-contrast  target  scenario  ‘otslcbds’. 
The  response  to  the  small  target  in  semd_array_otslcbds.avi  is  not  immediately  impressive 
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to  the  eye,  due  to  the  scintillation  of  the  activated  metapixels  and  the  presence  of  artifacts 
caused  by  background.  In  addition,  dropouts  occur  when  the  target  passes  over  nearly 
equiluminant  background.  Nevertheless,  if  this  video  is  stepped  frame-by-frame,  it  can  be 
seen  that  when  the  target  response  peaks,  it  is  typically  the  strongest  response  found  in  the 
entire  simulated  field  of  SEMDs.  This  selectivity  is  based  entirely  on  small  longitudinal 
size. 

In  spite  of  promising  results  in  a  variety  of  simulations,  the  SEMD  was  ultimately 
discounted  as  a  preliminary  elementary  motion-detecting  mechanism  in  the  proposed 
hierarchy  of  STMD  processing.  In  experimental  work,  when  biological  STMDs  are 
exposed  to  single  traveling  edges  (the  ‘extending  rod’  stimulus),  clear  responses  are 
obtained  when  the  edge  passes  through  the  STMD  receptive  field  (although  responses  are 
not  as  robust  as  for  small  objects  with  both  leading  and  trailing  edges).  This  result  is 
inconsistent  with  the  SEMD  because  the  model  depends  strictly  on  the  simultaneous 
passage  of  a  leading  edge  at  one  ommatidium  with  a  trailing  edge  of  opposite  polarity  at  a 
neighbor. 


3.1.3.  The  Primitive  Small  Target  Movement  Detector  (Shoemaker) 

As  noted,  the  PSTMD  model  is  intended  to  discriminate  target  from  background  motion 
on  the  basis  of  relative  velocity.  The  comparison  is  achieved  with  an  element  that  is 
excited  by  local  input,  and  is  also  subject  to  shunting  inhibition  by  a  delayed  signal  passed 
from  a  nearby  unit.  The  delayed  signal  represents  an  estimate  of  background  motion,  with 
the  delay  time  tuned  to  reflect  the  speed  of  that  motion.  Background  events  result  in 
suppression  of  response  by  the  shunting  inhibition,  whereas  events  inconsistent  with 
background  motion  allow  responses  to  ‘break  through’. 

Further  Characterization  of  the  Original  PSTMD 

Initially,  we  performed  simulations  of  the  PSTMD  using  the  algorithm  and  arrays 
developed  in  the  previous  effort,  but  with  inputs  supplied  by  SEMDs  rather  than 
correlational  EMDs  or  synthetic  input  signals  as  used  in  the  prior  simulations.  We  also 
examined  the  effectiveness  of  the  PSTMD  with  shorter-range  (nearest-neighbor) 
anticorrelations  rather  than  the  next-nearest-neighbor  basis  used  in  prior  simulations. 
Among  the  conclusions  reached  were  that  some  degree  of  smoothing  of  the  SEMD  output 
signals  (which  as  defined  above  may  have  discontinuous  first  temporal  derivatives)  is 
desirable  when  they  serve  as  inputs  to  the  PSTMD,  as  a  means  to  reduce  artifacts  in  the 
PSTMD  outputs.  These  artifacts  can  arise  due  to  the  interaction  of  the  sharp  changes  of  an 
undelayed  SEMD  signal  and  the  delayed  signal  which  is  smoothed  by  the  lowpass  filter 
operation  that  provides  the  delay.  Some  simulations  suggested  that  lowpass  filtering  of  the 
SEMD  output  with  a  first-order  filter  with  relatively  short  time  constant  (i.e.,  20ms)  could 
improve  performance.  A  second  conclusion  was  that  a  PSTMD  with  nearest-neighbor 
anticorrelations  does  not  allow  clear  discrimination  of  target  against  background  at  relative 
speeds  that  are  readily  discernible  by  the  human  observer.  However,  even  with  the  longer- 
range,  second-nearest-neighbor  interactions,  the  original  model  was  found  to  do  a 
generally  poor  job  of  distinguishing  target  from  background  motion  when  the  velocities  are 
at  all  similar.  As  a  result  we  undertook  a  major  revision  of  the  PSTMD  model. 


Tanner  Research  and  the  University  of  Adelaide 


17 


Qrant  FA9550-04- 1-0283 


Final  Report  31  DEC  2007 


Revision  of  the  PSTMD  with  ‘Propagating  Predictors  ’ 

This  revision  was  the  primary  accomplishment  on  the  present  project  with  respect  to  the 
PSTMD.  It  constitutes  an  elaboration  of  the  original  model  in  which  the  signal  estimating 
background  motion  was  generated  as  a  propagating  predictor  with  spatial  as  well  as 
temporal  memory.  A  schematic  diagram  illustrating  the  propagating  predictor  concept  is 
given  below  in  Figure  5: 


Figure  5:  ‘Propagating  predictor’  for  background  motion  in  a  PSTMD  model.  This 
example  is  for  a  unit  in  a  uniaxial  or  one-dimensional  array. 


This  diagram  depicts  one  element  of  a  one-dimensional  or  uniaxial  network,  with  a  pair  of 
connections  to  units  to  the  left  and  to  the  right,  and  with  a  local  input  presumed  to  come 
from  an  SEMD.  The  delay  operator  is  applied  to  the  both  the  local  input  {INPUT  FROM 
SEMD)  and  lateral  predictor  inputs  (IN  LEFT  and  IN  RIGHT).  The  comparison  of  the 
predictor  signal  to  the  input  signal  takes  place  after  application  of  the  delay,  so  that  the 
shape  (harmonic  content)  of  the  input  signal  is  similar  to  that  of  the  predictor  in  response 
to  a  moving  event.  Local  (additive)  inhibition  occurs  on  the  reciprocal  connections  ( OUT 
LEFT  and  OUT  RIGHT)  that  propagate  the  predictor  signal  from  the  element  to  its 
neighbors,  so  that  when  an  event  is  initially  propagated  from  a  neighbor  into  the  element, 
subsequent  propagation  from  the  element  back  to  that  neighbor  is  inhibited.  In  this  way,’ 
predictor  propagation  is  unidirectional  for  unidirectional  motion.  The  only  nonlinearity  in 
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the  predictor  network  is  the  half-wave  rectification  that  occurs  on  each  lateral  input.  The 
comparison  operation  between  the  predictor  and  (delayed)  input  signal  computed  by  this 
network  takes  place  by  the  same  shunting  inhibitory  mechanism  used  in  previous  PSTMD 
versions,  that  is  to  say,  division  by  the  predictor  signal  summed  with  a  small  constant  to 
prevent  singularity.  Tuning  of  the  delay  operator  to  the  speed  of  background  motion  was 
done  by  hand  in  simulations;  approaches  to  automatic  feedback  tuning  were  considered  but 
not  developed  beyond  the  conceptual  stage. 

Following  initial  simulations,  we  spent  some  time  analyzing  the  delay  operator  used  in 
the  PSTMD  model  in  the  context  of  its  input  signals,  which  are  assumed  to  come  from 
SEMDs.  The  SEMD  produces  only  positive  pulses  due  to  the  nature  of  its  processing,  and 
furthermore,  the  desired  delay  time  of  events  in  SEMD  output  space  tends  to  go  as  the 
duration  of  such  events,  since  they  represent  stimuli  in  which  a  leading  increase  in 
luminance  correlates  with  a  trailing  decrease,  or  vice-versa,  and  such  events  are  moving 
continuously  across  pairs  of  ommatidia.  One  conclusion  regarding  the  PSTMD  delay  is 
based  on  the  straightforward  observation  that  the  phase  delay  induced  by  a  linear  lowpass 
filter  increases  with  its  order,  while  for  a  given  phase  delay,  the  attenuation  and  change  in 
harmonic  content  of  the  output  relative  to  the  input  signal  decrease  with  increasing  order. 
Thus  a  high-order  filter  with  a  relatively  short  time  constant  is  desirable  to  implement  the 
delay  operator.  The  second  conclusion  is  that  it  is  desirable  to  implement  the  filter  with 
complex  poles,  which  improve  the  transient  response  time  and  decrease  attenuation  relative 
to  real  poles,  in  the  neighborhood  of  the  comer  frequency.  We  achieved  promising  results 
with  a  filter  order  of  six  (i.e.,  three  complex  pole-pairs)  and  with  a  quality  factor  Q  of  0.65- 
0.7  (with  Q=0.5  defining  the  limiting  case  of  real  poles).  Results  improve  if  the  order  is 
increased  beyond  six,  but  with  decreasing  returns  for  the  increased  computational 
complexity  (and  circuit  complexity  if  an  analog  VLSI  model  were  to  be  fabricated). 

Below  in  Figure  6  are  shown  the  results  for  a  one-dimensional  predictor  network  in 
response  to  data  taken  from  the  scenario  ‘otslcbds’: 
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Figure  6:  The  (filtered)  output  of  an  SEMD  (red)  and  a  propagating  predictor  (blue) 
at  one  location  in  a  one-dimensional  network.  Transit  of  a  small  target  moving  relative  to  the 
(moving)  background  occurs  just  prior  to  time  4s.  The  input  scenario  is  derived  from  one 
column  of  ‘otslcbds’,  and  the  target  is  moving  upward  during  transit.  • 


The  red  signal  is  the  output  of  an  SEMD  that  ‘sees’  the  target  just  before  time  =  4s.  Note 
the  SEMD  response  to  the  target  is  not  a  great  deal  larger  than  the  background  artifacts  in 
this  case.  The  blue  trace  is  the  propagating  predictor  output  at  the  same  location.  Note  the 
mis-registration  of  the  SEMD  and  predictor  responses  at  the  time  of  target  passage:  this 
results  in  a  significant  spike  in  the  PSTMD  output  (not  shown  here)  due  to  lack  of 
inhibition  of  the  SEMD  signal  during  its  rising  phase.  In  the  one-dimensional  case,  this 
scheme  was  found  to  give  good  speed-based  discrimination  for  all  of  our  orbiting-target 
input  scenarios. 

Unfortunately,  generalization  of  the  PSTMD  concept  to  two-dimensional  retinotopic 
space  proved  to  be  a  difficult  problem  due  to  the  introduction  of  direction  as  well  as  speed 
as  a  parameter  of  the  background  motion.  There  are  SEMD  units  aligned  with  each  of  the 
three  principal  directions  of  the  hexagonal  array,  but  the  predictor  cannot  simply  be 
restricted  to  one-dimensional  or  uniaxial  propagation  in  each  of  these  directions,  as  the 
predictor  signals  propagate  away  at  an  oblique  angle  from  the  direction  of  travel  when  a 
stimulus  traverses  the  network  in  a  direction  that  is  not  aligned  with  one  of  the  principal 
axes.  We  consequently  examined  approaches  allowing  propagation  of  predictor 
‘wavefronts’  in  two  dimensions.  This  approach,  however,  is  subject  to  the  dispersion  of 
energy  by  wave  propagation  in  directions  not  corresponding  to  target  motion.  We  hoped  to 
exploit  the  reinforcing  effect  of  constructive  interference  to  maximize  predictor  amplitude 
in  the  actual  direction  of  travel  (in  analogy  with  the  operation  of  Bartolac’s  ‘Wave  Process’ 
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mentioned  in  Section  2.1.6),  but  ultimately,  poor  predictor  performance  due  to  spurious 
propagation  could  not  be  overcome,  and  as  of  the  project’s  end  no  two-dimensional 
generalization  of  the  PSTMD  had  been  found  allowing  relative  velocity-based  target 
discrimination  remotely  approaching  that  of  the  one-dimensional  model. 

3. 1. 4.  The  Elementary  Small  Target  Motion  Detector  ( Shoemaker ,  O  ’Carroll) 

As  noted,  our  ESTMD  models  have  been  based  on  the  observed  characteristics  of  the 
class  of  neurons  that  we  have  also  labeled  ESTMDs.  The  basic  operation  of  our  model 
depends  on  a  receptive  field  in  retinotopic  space  with  a  notched  inhibitory  surround  and  an 
excitatory  center,  and  ‘tonic’  or  persistent  responses  in  each  region.  This  is  implemented 
with  the  following  functional  elements: 

1.  A  nonlinear  (fast  onset  /  slow  decay)  temporal  element.  This  displays  a  persistent 
response  following  an  input  stimulus. 

2.  A  diffusive  network  for  passive  electrotonic  spread  of  inhibition.  This  network 
receives  inputs  from  a  nonlinear  temporal  element. 

3.  A  summing  neuron  that  forms  a  receptive  field  with  excitatory  center  and  a  notched 
inhibitory  surround,  by  virtue  of  the  pattern  of  interconnection  weights  with  excitatory 
temporal  elements  and  with  the  inhibitory  network. 

In  the  intended  mode  of  operation,  only  when  a  target  enters  the  receptive  field  via  the 
notch,  and  is  small  enough  laterally  to  fit  through  the  notch,  can  the  summing  neuron  reach 
a  net  state  of  excitation.  If  a  target  enters  by  any  other  direction,  the  persistent  inhibition 
that  it  recruits  is  supposed  to  be  sufficient  to  preclude  a  net  excitatory  response  when  it 
reaches  the  excitatory  region. 

The  ESTMD  Temporal  Element  and  the  ‘Sustained  Tonic  ’  Response 

Initial  work  during  the  present  contract  focused  on  the  ‘persistent’  temporal  element. 
Near  the  close  of  the  prior  contract,  we  demonstrated  the  operation  of  the  nonlinear  or 
‘tonic’  temporal  filter  in  the  silicon  ESTMD  in  an  ‘overdriven’  mode.  In  this  mode, 
following  any  transient  input  excitation  that  exceeds  some  threshold  level,  the  temporal 
element  remains  fully  active  for  a  length  of  time  (determined  by  the  bias  of  the  filter 
circuit)  before  it  decays.  This  is  a  favorable  characteristic  for  purposes  of  modeling  the 
ESTMD  neuron,  allowing  more  robust  responsiveness  (i.e.,  adequate  function  over  a  wider 
range  of  target  characteristics  and  speeds).  The  ‘tonic’  operation  of  the  nonlinear  circuit 
previously  constituted  a  slow  constant  decay  of  post  excitation  response,  rather  than  any 
period  of  sustained  response.  We  developed  a  mathematical  behavioral  model  for  the 
‘sustained  tonic’  mode,  with  the  characteristics  (i.e.,  period  of  activity  and  decay  time) 
controllable  in  a  flexible  manner  by  a  relatively  few  parameters.  This  additional  flexibility 
could  be  readily  extended  to  the  circuit  with  some  simple  modifications.  The  series  of 
operations  can  be  summarized  as  a  clipping  or  squashing  nonlinearity  applied  to  the  input, 
followed  by  a  fast-onset,  slow-decay  nonlinear  filter,  followed  by  a  second  clipping 
operation,  at  a  lower  signal  amplitude  than  the  first.  The  governing  differential  equations 
are  given  in  (3): 

x  =  input;  z  =  output; 

y,  =  \/2(tanh[  g-(x-a)] +  \)  (3) 
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y2  =(exp[b(yx -y2)]-\)/z 
z  =  c ■  tanhf  y2  /c  ] 

where  g,  a,  b,  and  c  are  constants  with  g  and  a  chosen  based  on  the  input  signal 
characteristics,  c<l,  and  where  ris  a  time  constant  corresponding  to  persistence  time  of  the 
nonlinear  filter.  The  y/  and  y2  are  intermediate  variables  introduced  for  clarity,  and  a  dot 
over  a  variable  indicates  time  derivative. 

The  response  of  this  model  to  a  transient  input  is  illustrated  below  in  Figure  6: 


Figure  7:  Response  of  the  ‘sustained  tonic’  temporal  element.  The  blue  trace  is  the 
input  to  the  nonlinear  filter,  the  green  trace  is  the  output  of  the  filter,  and  the  red 
trace  is  the  final  output. 


Further  Development  of  the  SEMD  Model 

Following  development  of  the  sustained  tonic’  temporal  model,  we  made  several 
additional  changes  to  the  ESTMD  algorithm  and  topology  relative  to  the  previous  effort, 
prior  to  further  simulations.  First,  we  supplied  inputs  to  the  ESTMD  from  SEMDs  rather 
than  from  earlier  visual  stages.  (PSTMDs  were  not  included  due  to  lack  of  success  in 
formulating  two-dimensional  architectures.)  Rather  than  the  single  temporal  element  used 
in  prior  work,  we  began  to  use  two  elements  with  different  persistence  times  for  the 
excitatory  and  inhibitory  signals,  with  the  inhibitory  unit  assigned  a  longer  time  constant 
than  the  excitatory  unit.  One  such  pair  of  units  is  included  for  each  SEMD,  both  receiving 
input  from  the  SEMD.  Finally,  the  receptive  field  size  of  the  model  ESTMD  was 
expanded,  to  make  it  closer  to  those  of  biological  ESTMDs  (previous  models  had  a  single¬ 
pixel  excitatory  center).  The  revised  model  included  a  diffusive  inhibitory  network,  and  the 
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pooling  of  excitatory  and  inhibitory  inputs  was  modeled  with  a  linear  weighted  sum,  as  in 
previous  models. 

In  this  new  architecture,  connections  are  made  from  these  elements  onto  an  ESTMD 
‘neuron’  without  regard  to  the  orientations  of  the  underlying  SEMDs,  i.e.,  orientation 
selectivity  is  not  maintained  by  choice  of  SEMDs  which  feed  an  ESTMD  neuron.  Rather, 
orientation  selectivity  is  regained  -  and  direction  selectivity  established  -  by  the  spatial 
pattern  of  inter-connections  from  the  inhibitory  and  excitatory  elements  onto  the  ESTMD 
neuron.  The  particular  patter  of  connectivity  that  we  used  is  illustrated  below  in  Figure  8: 


Figure  8:  Depiction  of  interconnection  pattern  which  establishes  the  receptive  field  of 
a  model  ESTMD  neuron  in  retinotopic  space.  Hexagons  with  solid  boundaries 
indicate  ommatidia,  and  tetrahedrons  (diamonds)  with  dashed  boundaries  can  be 
regarded  as  indicating  SEMDs  as  well  as  elements  in  the  inhibitory  and  excitatory 
networks  that  operate  temporally  and  spatially  on  SEMD  outputs.  A  red  mark  at  the 
center  of  a  tetrahedron  indicates  a  connection  from  the  corresponding  excitatory 
element  onto  the  ESTMD  neuron  proper,  and  a  blue  mark  indicates  a  connection 
from  the  corresponding  inhibitory  element  onto  the  ESTMD  neuron. 
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This  pattern  gives  the  ESTMD  an  excitatory  receptive  field  area  of  about  three  degrees 
extent  (i.e.,  it  should  begin  to  reject  moving  stimuli  when  they  are  significantly  larger  than 
3  in  lateral  size),  inside  a  notched  inhibitory  receptive  field  area.  In  our  simulated  arrays, 
these  model  ESTMD  neurons  are  arranged  in  a  hexagonal  grid  in  retinotopic  space,  at 
twice  the  pitch  of  individual  ommatidia.  It  is  assumed  that  a  complete  system  would  have 
one  set  of  such  ESTMDs  aligned  with  each  of  the  six  principal  directions  of  the  hexagonal 
grid.  However,  we  generally  simulated  ESTMDs  with  a  single  preferred  direction,  and 
varied  the  direction  of  motion  of  visual  objects  in  order  to  investigate  directional 
selectivity. 

Initial  simulations  with  the  revised  ESTMD  model  were  aimed  at  developing  a  grasp  of 
the  response  characteristics  versus  expectations,  and  also  at  tuning  of  ESTMD  parameters. 
The  first  step  in  ESTMD  processing,  represented  by  the  first  of  Eqns.  (3),  provides  signal 
limiting  but  also  a  ‘soft  thresholding’  function  by  virtue  of  subtraction  of  the  threshold 
parameter  a  in  the  argument  of  the  tank.  Because  target  events  typically  (but  not  always) 
cause  larger  responses  in  the  SEMD  outputs  than  background  events  in  our  sequences,  we 
attempted  to  set  a  and  the  gain  g  such  that  most  artifacts  were  suppressed  while  target 
events  resulted  in  saturation.  In  the  nonlinear  ‘sustained  tonic’  filter  represented  by  the 
second  and  third  of  Eqns.  (3)  the  parameter  b  was  set  with  to  10  for  rapid  activation,  and 
the  persistence  time  x  was  set  to  100ms  for  the  excitatory  element  and  250ms  for  the 
inhibitory  element,  to  insure  that  inhibition  is  able  to  prevent  net  excitation  of  the  ESTMD 
neurons  when  a  target  passes  through  the  receptive  field  in  the  antipreferred  direction.  In 
addition,  the  shunt  and  lateral  conductances  in  the  electrotonic  inhibitory  network  were  set 
to  equal  values,  resulting  in  a  relatively  small  space  constant  (on  the  order  of  one  unit) 

Following  these  initial  simulations,  we  attempted  for  the  first  time  the  simulation  of 
complete  ESTMD  arrays  with  early  vision  and  SEMDs  as  precursor  processing.  Results 
were  moderately  encouraging.  We  did  expect  and  observed  a  significant  number  of 
artifacts  in  the  outputs  due  to  the  fact  that  our  input  data  contains  both  target  and 
background  motion,  and  the  precursor  processing  —  the  SEMD  —  has  no  relative  velocity- 
based  discrimination  capability  and  does  respond  to  the  motion  of  small  background 
features  as  well  as  targets.  In  addition  to  these  expected  artifacts,  we  also  observed  poorer- 
than-expected  rejection  of  larger  background  features. 

In  these  simulations,  the  receptive  fields  in  the  model  ESTMD  neurons  were  formed 
with  the  arbitrary  choice  of  unity  weights  for  all  excitatory  connections  depicted  in 
Figure  8,  and  weights  of  -1.5  for  the  inhibitory  connections.  We  performed  a  limited 
number  of  simulations  in  which  these  parameters  and  the  nonlinear  filter  parameters  were 
varied,  but  did  not  do  any  systematic  optimization  of  the  model. 

3.1.5.  Rectifying  Transient  Cells  (Shoemaker,  Wiederman) 

The  class  of  rectifying  transient  neurons  includes  neurons  that  in  the  past  have  been 
recorded  from  the  first  optic  chiasm,  representing  the  axons  of  afferent  neurons  projecting 
from  lamina  to  medulla  (in  fly,  Jansonius  and  van  Hateren,  1991,  1993a),  and  from  cells  in 
or  projections  into  the  medulla  (in  locust,  Osorio,  1987,  1991).  Recordings  from  blowflies 
in  the  O’Carroll  laboratory  appear  to  be  from  intrinsic  medulla  neurons,  although  it  is 
possible  they  may  be  from  cells  projecting  to  or  through  the  medulla  from  elsewhere  (i.e. 
from  laminar  RTCs). 
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The  published  and  observed  characteristics  of  RTCs,  and  our  experiences  in  modeling 
them,  have  given  us  for  the  first  time  (at  least  tentatively)  a  putative  precursor  element  on 
which  to  build  more  elaborated,  and  we  hope  better-performing,  STMD  models. 

Initial  Model  for  the  RTC 

Based  primarily  upon  previously  published  data  on  RTCs,  we  developed  an  initial 
computational  model  that  is  consistent  with  the  spatial  and  temporal  characteristics  of  the 
lamina  RTC  (the  so-called  ‘on-off  cell).  This  model  is  illustrated  below  in  schematic  form 
in  Figure  9: 
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Figure  9:  Block  diagram  depicting  a  model  for  the  processing  associated  with  a 
rectifying  transient  neuron,  analogous  to  the  lamina  ‘on-off  cell.  At  top  are  elements 
associated  with  a  single  optical  cartridge  up  to  the  level  of  individual  ‘ON’  and  ‘OFF’ 
neurons,  which  illustrates  the  origins  of  the  temporal  characteristics  of  the  cell.  At 
bottom  is  shown  the  interaction  between  and  summation  of  signals  from  ‘ON’  and 
‘OFF’  elements  in  multiple  adjacent  cartridges,  which  give  rise  to  the  spatial 
characteristics  of  the  receptive  field. 


In  this  model,  the  signal  path  leading  to  the  rectifying  transient  cell  is  divided  into 
distinct  ‘ON’  and  ‘OFF’  channels  at  a  relatively  early  stage.  In  this  regard,  it  is  similar  to  a 
detailed  model  for  the  lamina  on-off  cell  (Sarakaya,  Wang,  and  Ogmen,  1998)  but  differs 
from  other  prior  models  for  RTCs  (Jansonius  and  van  Hateren,  1993;  Osorio,  1991),  which 
have  placed  rectification  later  in  the  processing.  This  assumption  predicts  the  existence  of 
two  populations  of  lamina  neurons  to  carry  the  ON  and  OFF  signals;  Sarakaya  et  al.  (1998) 
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suggest  that  these  may  be  two  distinct  types  of  lamina  amacrine  cells,  but  we  note  that  the 
lamina  monopolar  cells  (LMCs)  and  the  ‘sustaining’  neurons  (tentatively  associated  with 
lamina  L5  cells,  Jansonius  and  van  Hateren  (1993))  have  transient  and  opposite-polarity 
responses  to  luminance  changes,  and  could  thus  be  part  of  the  two  pathways. 

In  our  models,  the  input  to  both  ON  and  OFF  channels  is  assumed  to  be  strongly 
highpass-filtered  in  the  temporal  domain  and  inverted  in  one  case,  then  half-wave  rectified. 
The  signals  in  the  two  channels  are  thus  transient  and  of  the  same  polarity,  although  they 
reflect  opposite  phases  of  the  input  waveform.  Adaptation  in  each  channel  is  achieved  by 
means  of  subtractive  inhibition.  The  input  signal  is  processed  by  an  element  (presently 
assumed  to  be  a  distinct  intemeuron)  with  fast  activation  and  slower  response  decay,  with 
an  output  corresponding  to  a  ‘state  of  adaptation’  for  that  channel.  This  output  is  subtracted 
from  the  original  signal  and  then  half-wave  rectified  by  a  subsequent  intemeuron,  so  that 
only  signal  levels  above  the  ‘state  of  adaptation’  are  passed  on.  Corresponding  to  this 
second  intemeuron  are  the  ‘ON’  and  ‘OFF’  intemeurons  in  the  top  part  of  Figure  9. 

The  spatial  characteristics  of  the  rectifying  transient  cell  are  modeled  by  assuming  that 
ON  and  OFF  intemeuron  outputs  are  summed  over  a  number  of  neighboring  cartridges. 
Prior  to  this  stage,  however,  lateral  inhibition  takes  place  between  intemeurons  of  each 
class,  leading  to  reduction  of  response  when  an  increasing  number  of  cartridges  are 
stimulated  with  luminance  changes  of  the  same  polarity,  as  observed  by  Jansonius  and  van 
Hateren  (1993a).  The  receptive  field  of  the  ‘  on-off”  cell  is  reported  by  Jansonius  and  van 
Hateren  (1993a)  to  have  a  spatial  extent  of  4  -  7  ommatidia. 

An  initial  version  of  this  model  for  simulation  purposes  was  built  from  abstract 
elements  in  the  SPICE  environment.  The  highpass  filter  at  top  in  Figure  9  is  composed  of 
linear  elements,  and  sums  and  differences  are  likewise  linear  operations  in  the  usual  sense. 
Signal  rectification  is  ideal  half-wave  rectification  (  POS(x)  =  (|x|+x)/2  ).  The  ‘fast 
activation,  slow  decay’  characteristic  used  to  compute  the  ‘state  of  adaptation’  for  an  ON 
or  OFF  channel  is  modeled  with  a  nonlinear  (diode)  conductance  charging  a  capacitance, 
which  then  discharges  through  a  linear  resistance. 

Modeling  the  RTC  with  ‘Neural  Primitives  ’ 

In  later  versions,  some  of  these  functions  were  modeled  in  a  way  that  makes  them  more 
consistent  with  actual  neuron  biophysics.  Interconnections  between  neuronal  analogs  are 
made  with  chemical  synapses’  that  comprise  a  variable  conductance  in  series  with  a 
voltage  source  representing  an  ionic  reversal  potential.  Summation  of  excitatory  and 
inhibitory  synaptic  currents  implements  ‘sums’  and  ‘differences’  (which  do  not  have  the 
usual  linear  sense  because  synaptic  currents  are  non-linearly  related  to  the  input  signals). 
The  fast  activation,  slow  decay  characteristic  is  achieved  naturally  with  an  intemeuron 
model  that  has  a  large  membrane  time  constant,  and  input  synapses  whose  modulated 
conductances  are  considerably  larger  than  the  intrinsic  membrane  conductance  for  typical 
inputs.  In  some  simulations,  a  multi-compartment  neuron  is  used  with  a  ‘dendritic’  cable 
interposed  between  the  synapses  and  output  site  (‘cell  body’).  Finally,  the  half-wave 
rectification  is  replaced  in  some  cases  by  a  smooth  bounded  strictly-positive  function  (i.e., 
a  sigmoid  whose  range  is  positive),  meant  to  represent  transformation  between  membrane 
potential  and  ‘output’  (either  firing  rate,  or  neurotransmitter  release  at  a  target  cell). 

This  work  was  done  in  the  SPICE  environment  that  has  been  used  for  simulations  to 
date  by  the  industrial  partner.  One  aim  of  developing  more  biophysically  realistic 
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processing  primitives  than  in  past  versions  was  to  initiate  work  on  a  general  ‘neural 
toolkit’  in  the  SPICE  environment  that  could  eventually  be  useful  (and  supplied  to  other 
interested  modelers  with  electronics  backgrounds)  for  multi-compartment  neural  modeling 
of  the  kind  that  is  now  performed  with  tools  such  as  GENESIS  and  NEURON.  Although 
such  a  ‘toolkit’  is  far  from  complete  at  this  stage,  an  initial  collection  of  useful  modules 
has  been  developed. 


Simulation  Results  for  RTC  Temporal  Characteristics 

Initial  simulations  focused  primarily  on  the  temporal  characteristics  of  the  RTC.  To 
give  a  qualitative  idea  of  the  capabilities  of  the  model,  below  we  compare  some 
preliminary  results  (Figure  1 1)  from  the  more  biologically-faithful  version  of  the  model  to 
results  obtained  by  Jansonius  and  van  Hateren  (1991)  and  Osorio  (1991)  in 
electrophysiological  experiments  testing  the  temporal  properties  of  RTCs  (Figure  10). 
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Figure  10:  Prior  electrophysiological  results  for  rectifying  transient  cells.  Parts  A-C 
are  for  blowfly  lamina  on-off  cells,  and  D  for  an  RTC  recorded  from  locust  medulla. 
Parts  A  and  B  show  rapid,  frequency-dependent  adaptation  to  luminance  pulses, 
corresponding  to  both  increases  (top)  and  decreases  (bottom)  in  intensity  from  a 
reference  level  /.  The  full-wave-rectifying  nature  of  the  response  is  clear.  Part  C 
illustrates  independence  of  adaptation  to  increases  and  decreases  in  luminance.  Part 
D  shows  how  increasing  the  intensity  of  a  periodic  pulse  stimulus  causes  a 
‘breakthrough’  of  the  response  from  adapted  levels,  or  alternatively,  a  ‘resetting  of 
the  contrast  threshold’  (Osorio,  1991). 
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Figure  11:  Simulated  responses  for  rectifying  transient  cells.  The  four  panels  at  left 
may  be  compared  with  electrophysiological  results  A,  B,  and  C  in  Figure  10.  The 
right-most  panel  may  be  compared  with  part  D  in  that  figure. 


3. 1. 6.  NMD  A  Receptors  and  Bistable  /Amplifying  Neural  Behavior 
(Shoemaker) 

Perhaps  the  most  significant  conclusion  of  the  study  of  NMDA  receptors  is  that 
bistability  mediated  by  these  receptors  requires  the  presence  of  additional  membrane 
conductance,  and  that  there  is  an  upper  bound  on  the  equivalent  reversal  potential 
associated  with  that  additional  conductance.  Using  the  Jahr-Stevens  (1991)  model  for 
NMDA  conductance  and  under  physiological  values  of  extracellular  [Mg2+],  this  maximum 
reversal  potential  value  is  found  to  be  about  -78mV.  This  implies  that  substantial 
potassium  conductance  must  be  present  to  enable  bistability.  The  only  major  synaptically- 
mediated  ion  channel  whose  primary  permeability  is  to  potassium  is  the  GABAs  receptor, 
and  so  much  of  the  analysis  considered  the  effects  of  coactivation  of  NMDA  and  GABAs 
synapses.  In  addition  to  a  maximum  condition  on  the  equivalent  reversal  potential,  limits 
(corresponding  to  fold  bifurcations)  were  computed  on  the  ratio  of  NMDA  to  total  ohmic 
conductance  compatible  with  bistability.  Outside  of  bistable  regimes,  amplifying  effects  of 
NMDA  receptors  were  noted,  and  maximum  gains  demonstrated  to  occur  again  with  co¬ 
occurrence  of  strongly  depolarizing  (i.e.,  potassium)  currents.  Dendrites  terminating  on  a 
somatic  load  resistance  were  analyzed  numerically,  and  bifurcation  between  mono-  and 
bistable  regimes  shown  to  occur  as  the  electrical  length  of  the  dendrite  is  varied,  in 
addition  to  bifurcations  with  variations  in  NMDA  and  depolarizing  conductances  in  the 
membrane.  Finally,  fast  AMPA  and  GABAb  inputs  were  shown  capable  of  triggering 
transitions  between  stable  states  when  a  compartment  or  dendrite  is  in  a  bistable  regime. 
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A  manuscript  detailing  the  results  of  the  study  was  prepared  and  submitted  to  PLoS 
Computational  Biology,  but  was  rejected.  It  is  presently  in  revision  and  will  be  resubmitted 
to  another  journal  near  or  shortly  after  termination  of  the  grant. 

3.1. 7.  Silicon  Modeling  (Shoemaker) 

On  the  DARPA  /  MIT/Lincoln  Labs  3DM2  run,  Tanner  Research  reserved  and  utilized 
space  corresponding  to  two  MOSIS  ‘tiny  chips’  (2mm  square  each),  each  with  three 
interconnected  vertical  tiers.  One  of  these  3-D  chips  contains  a  silicon  model  for  an  RTC, 
and  the  second  a  model  for  a  correlational  EMD.  The  parts  of  these  models  are  distributed 
among  the  tiers  as  follows.  The  first  tier  of  each  stacked  chip  contains  a  photodetector 
(with  design  guidance  provided  by  E.  Culurciello  at  Yale  University),  an  exponential 
(subthreshold  MOS)  load  for  the  photocurrents  to  provide  logarithmic  compression,  and  a 
first-order  lowpass  Gm-C  filter  as  an  initial  stage  for  subsequent  temporal  operations.  The 
second  tier  circuits  are  also  common  to  both  chips,  and  include  a  differential  circuit  to 
compute  a  highpass  output  as  the  difference  between  the  first-tier  raw  and  lowpass-filtered 
signals,  two  circuits  for  half-wave  rectification  of  each  waveform  phase,  a  nonlinear  filter 
(fast  rise,  ‘sustained  tonic’  response)  for  each  of  these  separate  ON  and  OFF  channels,  and 
a  shunting  inhibition  /  gain  control  stage  to  model  adaptation.  For  the  RTC  model,  the  third 
tier  contains  circuits  to  perform  spatial  inhibition,  and  to  recombine  the  ON  and  OFF 
channels.  For  the  EMD  model,  the  third  tier  contains  circuits  to  perform  a  delay  (lowpass 
filtering)  operation,  a  one-quadrant  log-domain  correlation,  and  to  recombine  signals 
differentially  to  obtain  a  bipolar  output.  This  EMD  model  is  novel  in  employing  the  RTC 
model  for  nonlinear  adaptation  as  a  model  for  motion  adaptation.  Schematic  diagrams  of 
the  circuits  are  depicted  below  in  Figure  12  -  Figure  15. 
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Figure  12:  First-tier  circuits,  including  photodiode,  exponential  load,  and  Gm-C  fdter, 
for  3-D  RTC  and  EMD  silicon  models. 
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Figure  13:  Second-tier  circuits,  including  ‘sustained  tonic’  filter  and  adjustable  gain 
circuits,  for  3-D  RTC  and  EMD  silicon  models. 
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Figure  14:  Third-tier  circuits  for  3-D  RTC  model. 
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Figure  15:  Third-tier  circuits  for  3-D  EMD  model. 


A  single  submission  of  two  test  die  containing  bio-inspired  circuits  was  made  to  a 
DARPA-sponsored  fabrication  run  of  an  experimental  three-dimensional  (three 
interconnected  interconnected  wafer  tiers)  integrated  circuit  process.  The  period  of 
performance  of  the  grant  was  extended  to  permit  receipt  and  testing  of  these  circuits,  but 
unfortunately  delays  in  the  completion  of  this  fabrication  have  precluded  test  and 
evaluation. 

4.  Personnel  Supported 

Patrick  Shoemaker,  Tanner  Research,  Inc.,  Co-Principal  Investigator 

Thomas  Bartolac,  Tanner  Research,  Inc.,  Senior  Scientist 

David  O’Carroll,  University  of  Adelaide,  Co-Principal  Investigator 

Karin  Nordstrom,  University  of  Adelaide,  Post-Doctoral  Fellow 

Steven  Wiederman,  University  of  Adelaide,  PhD  Student 

Paul  Barnett,  University  of  Adelaide,  PhD  Student 

Douglas  Bolzon,  University  of  Adelaide,  PhD  Student 

Bart  Geurten,  Diploma  (exchange)  student  from  University  of  Koln,  Germany 

Russell  Brinkworth,  University  of  Adelaide,  Post-Doctoral  Fellow 

Lachlan  Dowd,  University  of  Adelaide,  Computer  Support  Officer 

5.  Publications 

5.1. Papers  Produced  Under  Support  of  this  Grant: 

Shoemaker,  P.  A  methodology  for  long  time-constant  log-domain  filters  in  CMOS. 
Analog  Integrated  Circuits  and  Signal  Processing ,  Vol.  42,  pp.  161-178,  2005.  (Produced 
under  prior  contract  but  appearing  in  print  during  the  term  of  this  grant.) 

Nordstrom,  K.,  and  O’Carroll,  D.C.  Insect  detection  of  small  targets  moving  in  visual 
clutter.  PLoS  Biology  vol.  4  (3),  pp.  378-386,  2006.  DOI:  10.1371/joumal.pbio.0040054 
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Nordstrom,  K.,  and  O’ Carroll,  D.C.  Small  target  detection  neurons  in  female 
hoverflies.  Proceedings  of  the  Royal  Society  of  London  B,  vol.  273  (1591),  pp.  121 1-1216 
2006.  DOI:  rspb.2005.3424 

O’Carroll,  D.C.,  Barnett,  P.D.,  Mah,  E.L.  Nordstrom,  K  and  Brinkworth,  R.S.A.  A 
neuromorphic  model  for  a  robust,  adaptive  photoreceptor  reduces  variability  in  correlation 
based  motion  detectors.  BICS  06,  Brain  Inspired  Cognitive  Systems,  2006. 

Barnett,  P.D.,  Nordstrom,  K  and  O’Carroll,  D.C.,  Retinotopic  Organization  of  Small- 
Field  Target  Detecting  Neurons  in  the  Insect  Visual  System.  Current  Biology  Vol  17  dd 
569-578,2007. 

Guerten,  B.R.H.,  Nordstrom,  K.,  Sprayberry,  J.D.H.,  Bolzon,  D.,  and  O’Carroll,  D.C., 
Neural  mechanisms  underlying  target  detection  in  a  dragonfly  centrifugal  neuron.  Journal 
of  Experimental  Biology  vol.  210,  pp.  3277-3284,  2007. 

Shoemaker,  P.A.  Analysis  of  neural  bistability  and  amplification  mediated  by  NMD  A 
receptors.  In  preparation/revision,  2007. 

5.2.  Papers  by  the  Principals  in  Related  Areas 

Shoemaker,  P.A.,  O  Carroll,  D.C.,  and  Straw,  A.D.  (2005)  Velocity  constancy  and 
models  for  wide-field  motion  detection  in  insects.  Biological  Cybernetics,  Vol  93  No  4 
pp.  275-287,  2005. 

Guzinski,  R.,  Npiyen,  K.,  Yong,  Z.H.,  Rajesh,  S.,  O'Carroll,  D.C.,  and  Abbott,  D. 
Characterization  of  insect  vision  based  collision  avoidance  models  using  a  video  camera. 
Proceedings  SPIE  Vol.  6036,  pp.  328-342,  2006. 

Rajesh,  S.,  Abbott,  D.,  and  O'Carroll,  D.C.  A  16  pixel  yaw  sensor  for  velocity 
estimation.  Proceedings  SPIE  vol.  6036,  pp.  309-318,  2006. 

Mah,  E.L.,  Brinkworth,  R.S.,  and  O'Carroll,  D.C.  Bio-inspired  analog  circuitry  model 
of  insect  photoreceptor  cells.  Proc.  SPIE  Vol.  6036,  pp.  280-291,  2006. 

Straw,  A.D.,  Warrant,  E.J.,  and  O  Carroll,  D.C.  A  ‘bright  zone’  in  male  hoverfly 
(Eristalis  tenax)  eyes  and  associated  faster  motion  detection  and  increased  contrast 
sensitivity.  Journal  of  Experimental  Biology  Vol.  209,  pp.  4339-4354,  2006. 

O’Carroll,  D.C.,  Shoemaker,  P.A,  and  Brinkworth,  R.S.A.  Bioinspired  optical  rotation 
sensor.  Proceedings  SPIE,  vol.  6414,  p.  641418,  2007. 

Brinkworth,  R.S.A.,  Mah,  E.L.,  and  O’Carroll,  D.C.  Pixel-wise  adaptive  imaging. 
Proceedings  SPIE  Vol.  6414,  pp.  641418,  2007. 

6.  Interactions  /  Transitions 

6.1.  Participation/presentation  at  meetings,  conferences,  and  seminars 

Karin  Nordstrom  attended  the  Congress  of  the  International  Society  for  Neuroethology, 
8-13  August  2004,  Nyborg,  Denmark. 

David  O’Carroll  and  Karin  Nordstrom  attended  the  International  Congress  of 
Entomology,  16-21  August  2004,  Brisbane,  Australia. 

Patrick  Shoemaker,  David  O’ Carroll,  and  Karin  Nordstrom  attended  the  Workshop  on 
Insect  Sensors  and  Robotics,  23-26  August  2004,  Brisbane,  Australia. 
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Patrick  Shoemaker,  David  O’Carroll,  Steven  Wiederman,  and  Paul  Barnett  attended  the 
inaugural  Queensland  Brain  Institute  Workshop  on  Computational  and  Mathematical 
Neuroscience,  13-14  August  2006,  Brisbane,  Australia. 

6.1.1.  Conference  Presentations  / Posters  Supported  Directly  by  this  Grant: 

Nordstrom,  K.,  O  Carroll,  D.C.  Small  target  motion  detector  neurons  are  present  in 
female  hoverflies.  Proceedings  of  the  Seventh  Congress  of  the  International  Society  for 
Neuroethology,  Nyborg,  Denmark,  8-13  August  2004. 

Nordstrom,  K.,  O’Carroll,  D.C.  Characterization  of  small  target  motion  detection 
neurons  of  Eristalis  tenax.  Proceedings  of  the  XXII  International  Congress  of  Entomology, 
Brisbane,  Australia,  23-26  August  2004. 

Nordstrom,  K..,  Shoemaker,  P .A.  O  Carroll,  D.C.  Detection  of  small  low-contrast 
targets  across  cluttered  and  moving  backgrounds.  Workshop  on  Insect  Sensors  and 
Robotics,  Brisbane,  Australia,  23-26  August  2004. 

Nordstrom,  K.,  and  O  Carroll,  D.C.  Asymmetric  receptive  fields  of  target  neurons  in 
the  dragonfly  lobula.  Society  for  Experimental  Biology  Annual  Main  Meeting  2005, 
Barcelona,  Spain. 

Shoemaker,  P.,  Weiderman,  S.,  and  O’Carroll,  D.C.  Nonlinear  neurons  in  the  early 
visual  ganglia  of  insects:  new  results  and  a  computational  model.  Inaugural  Queensland 
Brain  Institute  Workshop  on  Mathematical  and  Computational  Neuroscience,  2006, 
Brisbane,  Australia. 

Weiderman,  S.,  and  O’Carroll,  D.C.  Modeling  rectified  transient  neurons  as  inputs  to 
correlation-type  motion  detection  mechanisms.  Inaugural  Queensland  Brain  Institute 
Workshop  on  Mathematical  and  Computational  Neuroscience,  2006,  Brisbane,  Australia. 

Nordstrom,  K.,  Barnett,  P.D.,  and  O’Carroll,  D.C.  Visual  target  detection  in  clutter. 
Australasian  Society  for  the  Study  of  Animal  Behaviour  Conference,  2006,  Sydney 
Australia. 


6. 1.2.  Conference  Presentations  /  Posters  by  the  Principals  in  Related  A reas: 

Nordstrom,  K,  Swan,  L.,  Mayrhofer,  S.,  O’Carroll,  D.C.  Comparison  of  the  image  step 
response  in  flies  and  humans.  Proceedings  of  the  XXII  International  Congress  of 
Entomology,  Brisbane,  Australia,  16-21  August  2004. 

Shoemaker,  P.A.,  O’Carroll,  D.C.  Silicon  and  abstract  modeling  of  adaptive  motion 
detection.  Workshop  on  Insect  Sensors  and  Robotics,  Brisbane,  Australia  23-26  August 
2004. 

O’Carroll,  D.C.,  Shoemaker,  P.A.,  Rainsford,  T.  ,  Straw,  A.D.,  Mah,  E.L.,  Rajesh,  S. 
Motion  detection  and  velocity  coding  of  natural  scenes  by  the  insect  visual  system. 
Workshop  on  Insect  Sensors  and  Robotics,  Brisbane,  Australia,  23-26  August  2004. 

Rainsford,  T.,  O’Carroll,  D.C.  Pattern  noise  in  neural  coding  of  natural  moving  images. 
Workshop  on  Insect  Sensors  and  Robotics,  Brisbane,  Australia,  23-26  August  2004. 

O  Carroll,  D.C.,  Rainsford,  T.,  Mah,  E.L.,  Rajesh,  S.  A  16-pixel  yaw  sensor  based  on 
insect  motion  detection.  Workshop  on  Insect  Sensors  and  Robotics,  Brisbane,  Australia 
23-26  August  2004. 

Rajesh,  S.,  O’Carroll,  D.C.  Elaborated  Reichardt  models  for  the  detection  of  optical 
flow.  Workshop  on  Insect  Sensors  and  Robotics,  Brisbane,  Australia,  23-26  August  2004. 
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Mah,  E.L.,  O’Carroll,  D.C.  Implementation  of  biomimetic  models  for  adaptive 
photoreceptors  based  on  insect  neurobiology.  Workshop  on  Insect  Sensors  and  Robotics, 
Brisbane,  Australia,  23-26  August  2004. 

Dowd,  L.M.,  Ali,  I.,  Straw,  A.D.,  O’Carroll,  D.C.  Interfacing  VisionEgg-based 
stimulus  generators  for  testing  responses  of  biological  and  artificial  vision  systems. 
Workshop  on  Insect  Sensors  and  Robotics,  Brisbane,  Australia,  23-26  August  2004. 

Moyer  de  Miguel,  I.,  Nordstrom,  K.,  O’Carroll,  D.C.  Local  motion  adaptation  in  insect 
visual  systems.  Society  for  Experimental  Biology  Annual  Main  Meeting,  Barcelona,  Spain 
2005. 


6.1.3.  Participation  in  Seminars  and  Laboratory  Meetings 

Electrophysiology  of  Eristalis  target  neurons.  Talk  given  at  Department  of  Zoology, 
University  of  Cambridge,  UK,  by  K.  Nordstrom,  2005. 

Sexual  dimorphism  of  hoverfly  target  detection.  Talk  given  at  Department  of 
Neuroscience,  School  of  Biology,  University  of  Bielefeld,  Germany,  by  K.  Nordstrom 
2005. 

Hoverfly  detection  of  small  targets.  Talk  given  at  Visual  Sciences,  Research  School  of 
Biological  Sciences,  Australian  National  University,  by  K.  Nordstrom,  2005. 

Hoverfly  motion  vision.  Talk  given  at  Nobel  Institute  for  Neurophysiology,  Karolinska 
Institute,  Sweden,  by  K.  Nordstrom,  2005. 

Motion  vision  in  hoverflies.  Talk  given  at  Department  of  Neuroscience,  Uppsala 
University,  Sweden,  K.  Nordstrom,  2005. 

Motion  vision  during  flight  -  exciting  new  hoverfly  findings.  Talk  given  at  Discipline 
of  Physiology,  The  University  of  Adelaide,  K.  Nordstrom,  2005. 

Rectifying  transient  cell  model.  Talk  given  at  Discipline  of-Physiology,  The  University 
of  Adelaide,  S.  Wiederman,  2006. 

Insect  visual  motion  detection  and  autonomous  navigation.  Talk  given  at  Discipline  of 
Physiology,  The  University  of  Adelaide,  P.  Shoemaker,  2007. 

Analysis  of  neural  bistability  and  amplification  mediated  by  NMDA  receptors.  Talk 
given  at  the  Laboratory  for  Neural  Computation,  The  University  of  Southern  California,  P. 
Shoemaker,  2007. 

6.1.4.  Other 

In  an  agreement  with  the  DARP A/Lincoln  Laboratory  3D-IC  program  managers, 
Patrick  Shoemaker  developed  technology  files  (including  layout  setup,  connectivity  rules, 
and  basic  design  rules)  for  the  3DM2  process  in  the  Tanner  Research  IC  design  tool  suite, 
in  exchange  for  no-cost  access  to  a  DARPA-sponsored  3D  IC  fabrication  run.  These  files 
and  associated  documentation  were  made  available  to  the  program  managers  and  to  other 
participants  in  the  program  who  used  Tanner  tools  on  their  projects,  including  R.  Lovejoy 
at  Sandia  National  Labs,  J.  Osborn  at  the  Aerospace  Corporation,  A.  Andreou  at  Johns 
Hopkins  University,  M.  Zalalutdinov  at  NRL,  and  the  faculty  and  staff  at  North  Carolina 
State  University  who  produced  the  primary  tool  suite  for  the  process.  Shoemaker  assisted 
these  participants  with  technology  file  installation  and  use,  and  incorporated  feedback  in 
revisions  of  the  files  during  the  course  of  the  design  phase. 
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In  addition,  Shoemaker  consulted  with  E.  Culurciello  of  Yale  University,  who  kindly 
provided  design  guidance  for  the  fabrication  of  photodetectors  in  the  thin-film  silicon-on- 
insulator  3  DM2  technology. 

6.2.  Consultative  and  advisory  functions 

Patrick  Shoemaker  is  serving  as  a  co-advisor  on  the  Ph.D.  of  Steven  Wiederman  at  the 
University  of  Adelaide. 

6.3.  Transitions 

As  a  result  of  work  under  this  grant,  Karin  Nordstrom  secured  a  Fellowship  via  the 
Swedish  Research  Council,  which  from  31  January  2005  has  paid  a  large  part  of  her  salary. 
This  freed  up  resources  to  pay  Lachlan  Dowd  for  programming  support  to  the  project. 

Tanner  Research  was  awarded  two  Phase  I  SBIR  contracts,  Navy  contract  N68335-06- 
C-0211  and  Air  Force  contract  FA8651-06-M-0203,  and  a  follow-on  Air  Force  Phase  II 
contract,  FA8651-07-C-0099.  These  deal  with  applications  of  wide-field  visual  motion 
detection,  but  have  made  use  of  models  developed  under  support  of  the  present  grant 
(specifically,  using  rectifying  transient  cell  models  as  drivers  for  motion  adaptation). 
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7.  New  Discoveries,  Inventions,  or  Patent  Disclosures 

The  discovery  of  STMD  neurons  in  female  hoverflies  by  Nordstrom  and  O’Carroll  was 
a  novel  finding,  leading  to  publication  of  a  paper  in  the  Proceedings  of  the  Royal  Society 
of  London. 

An  (apparently)  novel  semiconductor  device,  a  dual  complementary  channel  MOS 
transistor  realizable  in  silicon-on-insulator  technology,  was  conceived  by  Shoemaker  and 
placed  on  the  DARPA-sponsored  3DM2  run  by  Lincoln  Laboratories.  Due  to  delay  in 
completion  of  this  run,  the  circuit  is  not  available  for  test  and  evaluation  before  the 
termination  of  the  grant;  characterization  and  a  literature  survey  for  similar  inventions  will 
take  place  in  subsequent  efforts. 

8.  Honors  /  Awards 

For  travel  to  conferences  and  foreign  laboratories,  Karin  Nordstrom  received  a  School 
ECR  travel  award,  a  Health  Sciences  Travel  award,  and  an  Oxford  Gordon  Conference 
travel  award  during  the  grant. 
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